Learning Dual Semantic Relations With Graph Attention for Image-Text Matching

Image-Text Matching is one major task in cross-modal information processing. The main challenge to learn the unified visual and textual representations. Previous methods that perform well on this primarily focus not only alignment between region features images corresponding words sentences, but also relations of regions relational words. However, lack joint learning regional global will cause lose contact with context, leading mismatch those non-object which have meanings some sentences. In work, order alleviate issue, it necessary enhance concepts obtain a more accurate representation so as be better correlated text. Thus, novel multi-level semantic enhancement approach named Dual Semantic Relations Attention Network(DSRAN) proposed mainly consists two modules, separate module module. DSRAN performs graph attention both modules respectively for region-level regional-global at same time. With these different hierarchies are learned simultaneously, thus promoting image-text matching process by providing final representation. Quantitative experimental results been performed MS-COCO Flickr30K our method outperforms previous approaches large margin due effectiveness dual scheme. Codes available https://github.com/kywen1119/DSRAN.